<!DOCTYPE html>
<html lang="zh-cn">
    <head><meta charset='utf-8'>
<meta name='viewport' content='width=device-width, initial-scale=1'>
<meta name="baidu-site-verification" content="code-N4HqDA2yr7" />
<meta name="msvalidate.01" content="C30AE7336C0C214E141AC1AAC39A9A05" /><meta name='description' content='为什么使用cpp新建op 一些操作表示成现有操作的组合不好实现或者无法实现。 已有操作的组合效率不高。 想要自定义一些基本操作的组合，因为未来编译'><title>tensorflow新建op (cpp) </title>

<link rel='canonical' href='/tensorflow/tensorflow-custom-op-cpp/'>

<link rel="stylesheet" href="/scss/style.min.732a6bd9762caf6dce004132442f09bcd64101bf70fd8ab25f9c10d85d58f6b7.css"><meta property='og:title' content='tensorflow新建op (cpp) '>
<meta property='og:description' content='为什么使用cpp新建op 一些操作表示成现有操作的组合不好实现或者无法实现。 已有操作的组合效率不高。 想要自定义一些基本操作的组合，因为未来编译'>
<meta property='og:url' content='/tensorflow/tensorflow-custom-op-cpp/'>
<meta property='og:site_name' content='倔强的贝吉塔'>
<meta property='og:type' content='article'><meta property='article:section' content='Tensorflow' /><meta property='article:tag' content='tensorflow1.x' /><meta property='article:tag' content='op' /><meta property='article:published_time' content='2022-08-06T22:12:00&#43;08:00'/><meta property='article:modified_time' content='2022-08-06T22:12:00&#43;08:00'/><meta property='og:image' content='/categories/tensorflow/tensorflow.png' />
<meta name="twitter:title" content="tensorflow新建op (cpp) ">
<meta name="twitter:description" content="为什么使用cpp新建op 一些操作表示成现有操作的组合不好实现或者无法实现。 已有操作的组合效率不高。 想要自定义一些基本操作的组合，因为未来编译"><meta name="twitter:card" content="summary_large_image">
    <meta name="twitter:image" content='/categories/tensorflow/tensorflow.png' />
    <link rel="shortcut icon" href="/img/sorcerer.png" />

<script async src="https://www.googletagmanager.com/gtag/js?id=G-V4H6ZTRWB4"></script>
<script>
var doNotTrack = false;
if (!doNotTrack) {
	window.dataLayer = window.dataLayer || [];
	function gtag(){dataLayer.push(arguments);}
	gtag('js', new Date());
	gtag('config', 'G-V4H6ZTRWB4', { 'anonymize_ip': false });
}
</script>

            
<script async src="https://www.googletagmanager.com/gtag/js?id=G-V4H6ZTRWB4"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  gtag('config', 'G-V4H6ZTRWB4');
</script>

    </head>
    <body class="
    article-page has-toc
">
    <script>
        (function() {
            const colorSchemeKey = 'StackColorScheme';
            if(!localStorage.getItem(colorSchemeKey)){
                localStorage.setItem(colorSchemeKey, "dark");
            }
        })();
    </script><script>
    (function() {
        const colorSchemeKey = 'StackColorScheme';
        const colorSchemeItem = localStorage.getItem(colorSchemeKey);
        const supportDarkMode = window.matchMedia('(prefers-color-scheme: dark)').matches === true;

        if (colorSchemeItem == 'dark' || colorSchemeItem === 'auto' && supportDarkMode) {
            

            document.documentElement.dataset.scheme = 'dark';
        } else {
            document.documentElement.dataset.scheme = 'light';
        }
    })();
</script>
<div class="container main-container flex 
    
        extended
    
">
    
        <div id="article-toolbar">
            <a href="/" class="back-home">
                <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-chevron-left" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
  <path stroke="none" d="M0 0h24v24H0z"/>
  <polyline points="15 6 9 12 15 18" />
</svg>



                <span>返回</span>
            </a>
        </div>
    
<main class="main full-width">
    <article class="has-image main-article">
    <header class="article-header">
        <div class="article-image">
            <a href="/tensorflow/tensorflow-custom-op-cpp/">
                
                    <img src="/categories/tensorflow/tensorflow.png" loading="lazy" alt="Featured image of post tensorflow新建op (cpp) " />
                
            </a>
        </div>
    

    <div class="article-details">
    
    <header class="article-category">
        
            <a href="/categories/tensorflow/" >
                tensorflow
            </a>
        
    </header>
    

    <h2 class="article-title">
        <a href="/tensorflow/tensorflow-custom-op-cpp/">tensorflow新建op (cpp) </a>
    </h2>

    

    
    <footer class="article-time">
        
            <div>
                <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-calendar-time" width="56" height="56" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
  <path stroke="none" d="M0 0h24v24H0z"/>
  <path d="M11.795 21h-6.795a2 2 0 0 1 -2 -2v-12a2 2 0 0 1 2 -2h12a2 2 0 0 1 2 2v4" />
  <circle cx="18" cy="18" r="4" />
  <path d="M15 3v4" />
  <path d="M7 3v4" />
  <path d="M3 11h16" />
  <path d="M18 16.496v1.504l1 1" />
</svg>
                <time class="article-time--published">2022-08-06</time>
            </div>
        

        
            <div>
                <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-clock" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
  <path stroke="none" d="M0 0h24v24H0z"/>
  <circle cx="12" cy="12" r="9" />
  <polyline points="12 7 12 12 15 15" />
</svg>



                <time class="article-time--reading">
                    阅读时长: 4 分钟
                </time>
            </div>
        
    </footer>
    
</div>
</header>

    <section class="article-content">
    
    
    <h2 id="为什么使用cpp新建op">为什么使用cpp新建op</h2>
<ol>
<li>一些操作表示成现有操作的组合不好实现或者无法实现。</li>
<li>已有操作的组合效率不高。</li>
<li>想要自定义一些基本操作的组合，因为未来编译器做这种融合可能会比较困难。</li>
</ol>
<h2 id="如何使用cpp新建op">如何使用cpp新建op</h2>
<ol>
<li>注册op，注册op会定义一个接口（规范），比如定义op的名称和它的输入输出、shape函数（用于获取张量的形状）</li>
<li>实现op，对于CPU和GPU可以有不同的实现</li>
<li>为op编写一个函数来计算梯度（可选）</li>
</ol>
<p>其实通过前两个步骤，我们就可以编写出一个可用的op，只是神经网络的反向传播需要计算梯度，因此涉及到反向传播求梯度操作时，我们还需要为op编写梯度计算函数，如果自定义的op不涉及求梯度，则无需编写梯度计算函数。</p>
<h2 id="新建矩阵乘法op">新建矩阵乘法op</h2>
<h3 id="注册矩阵乘法op">注册矩阵乘法op</h3>
<p>您可以在 <a class="link" href="https://github.com/StubbornVegeta/tensorflow-custom-op"  target="_blank" rel="noopener"
    ><strong>这里</strong></a> 看到完整的代码</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-cpp" data-lang="cpp"><span class="line"><span class="cl"><span class="cp">#include</span> <span class="cpf">&#34;tensorflow/core/framework/op_kernel.h&#34;</span><span class="cp">
</span></span></span><span class="line"><span class="cl"><span class="cp">#include</span> <span class="cpf">&#34;tensorflow/core/framework/op.h&#34;</span><span class="cp">
</span></span></span><span class="line"><span class="cl"><span class="cp">#include</span> <span class="cpf">&#34;tensorflow/core/framework/shape_inference.h&#34;</span><span class="cp">
</span></span></span><span class="line"><span class="cl"><span class="cp"></span>
</span></span><span class="line"><span class="cl"><span class="k">using</span> <span class="k">namespace</span> <span class="n">tensorflow</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">REGISTER_OP</span><span class="p">(</span><span class="s">&#34;Mymatmul&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">.</span><span class="n">Attr</span><span class="p">(</span><span class="s">&#34;T: {float, int32, int64, double}&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">.</span><span class="n">Input</span><span class="p">(</span><span class="s">&#34;matrix1: T&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">.</span><span class="n">Input</span><span class="p">(</span><span class="s">&#34;matrix2: T&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">.</span><span class="n">Output</span><span class="p">(</span><span class="s">&#34;matmuled: T&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">.</span><span class="n">SetShapeFn</span><span class="p">([](</span><span class="o">::</span><span class="n">tensorflow</span><span class="o">::</span><span class="n">shape_inference</span><span class="o">::</span><span class="n">InferenceContext</span><span class="o">*</span> <span class="n">c</span><span class="p">){</span>
</span></span><span class="line"><span class="cl">                <span class="k">auto</span> <span class="n">N</span> <span class="o">=</span> <span class="n">c</span><span class="o">-&gt;</span><span class="n">Dim</span><span class="p">(</span><span class="n">c</span><span class="o">-&gt;</span><span class="n">input</span><span class="p">(</span><span class="mi">0</span><span class="p">),</span> <span class="mi">0</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">                <span class="k">auto</span> <span class="n">M</span> <span class="o">=</span> <span class="n">c</span><span class="o">-&gt;</span><span class="n">Dim</span><span class="p">(</span><span class="n">c</span><span class="o">-&gt;</span><span class="n">input</span><span class="p">(</span><span class="mi">1</span><span class="p">),</span> <span class="mi">1</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">                <span class="n">c</span><span class="o">-&gt;</span><span class="n">set_output</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">c</span><span class="o">-&gt;</span><span class="n">MakeShape</span><span class="p">({</span><span class="n">N</span><span class="p">,</span><span class="n">M</span><span class="p">}));</span>
</span></span><span class="line"><span class="cl">                <span class="k">return</span> <span class="n">Status</span><span class="o">::</span><span class="n">OK</span><span class="p">();</span>
</span></span><span class="line"><span class="cl">              <span class="p">});</span>
</span></span></code></pre></div><ul>
<li>这里通过<code>.Attr</code>为输入输出添加多种类型，从而达到多态的目的</li>
<li>我们可以从<code>InferenceContext*</code>类型的上下文参数中获取输入以及它们的形状</li>
<li><code>SetShapeFn</code>用来确定输出的形状
<ul>
<li><code>c-&gt;input(0)</code>： 获取第一个输入参数</li>
<li><code>c-&gt;Dim(X, 0)</code>：获取<code>X</code>的第一个维度</li>
<li><code>c-&gt;set_output(0, ...)</code>：设置第一个输出的形状</li>
</ul>
</li>
</ul>
<h3 id="实现op">实现op</h3>
<p>实现op的大体框架如下：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-cpp" data-lang="cpp"><span class="line"><span class="cl"><span class="k">template</span><span class="o">&lt;</span><span class="k">typename</span> <span class="n">T</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl"><span class="k">class</span> <span class="nc">MymatmulOp</span> <span class="o">:</span> <span class="k">public</span> <span class="n">OpKernel</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"><span class="k">public</span><span class="o">:</span>
</span></span><span class="line"><span class="cl">  <span class="k">explicit</span> <span class="n">MymatmulOp</span><span class="p">(</span><span class="n">OpKernelConstruction</span><span class="o">*</span> <span class="n">context</span><span class="p">)</span> <span class="o">:</span> <span class="n">OpKernel</span><span class="p">(</span><span class="n">context</span><span class="p">)</span> <span class="p">{}</span>
</span></span><span class="line"><span class="cl">  <span class="kt">void</span> <span class="nf">Compute</span><span class="p">(</span><span class="n">OpKernelContext</span><span class="o">*</span> <span class="n">context</span><span class="p">)</span> <span class="k">override</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="c1">// ...
</span></span></span><span class="line"><span class="cl"><span class="c1"></span>  <span class="p">}</span>
</span></span></code></pre></div><p>我们要创建一个继承自<code>OpKernel</code>的类，并重载<code>Compute</code>方法</p>
<p><code>Compute</code>方法有一个类型为<code>OpKernelContext*</code>的参数<code>context</code>，从中可以访问输入输出张量等有用的信息</p>
<p>接下来我们要在这个框架中完成具体的op实现</p>
<h4 id="创建输入输出张量">创建输入输出张量</h4>
<p>我们可以从<code>context</code>中直接读取输入张量以及它们的形状，根据它们的形状来计算输出张量的形状，进而为输出张量分配内存:</p>
<p>$$ Output_{N\times M} = Input1_{N\times K} \times Input2_{K\times M}  $$</p>
<p>$$ \downdownarrows $$</p>
<p>$$[N, K] \times [K, M] \to [N, M]$$</p>
<p>具体实现如下：</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-cpp" data-lang="cpp"><span class="line"><span class="cl">    <span class="c1">// create input tensor
</span></span></span><span class="line"><span class="cl"><span class="c1"></span>    <span class="k">const</span> <span class="n">Tensor</span><span class="o">&amp;</span> <span class="n">input_tensor1</span> <span class="o">=</span> <span class="n">context</span><span class="o">-&gt;</span><span class="n">input</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">    <span class="k">const</span> <span class="n">Tensor</span><span class="o">&amp;</span> <span class="n">input_tensor2</span> <span class="o">=</span> <span class="n">context</span><span class="o">-&gt;</span><span class="n">input</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">    <span class="k">const</span> <span class="n">TensorShape</span><span class="o">&amp;</span> <span class="n">input1_shape</span> <span class="o">=</span> <span class="n">input_tensor1</span><span class="p">.</span><span class="n">shape</span><span class="p">();</span>
</span></span><span class="line"><span class="cl">    <span class="k">const</span> <span class="n">TensorShape</span><span class="o">&amp;</span> <span class="n">input2_shape</span> <span class="o">=</span> <span class="n">input_tensor2</span><span class="p">.</span><span class="n">shape</span><span class="p">();</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1">// create output tensor
</span></span></span><span class="line"><span class="cl"><span class="c1"></span>    <span class="n">TensorShape</span> <span class="n">output_shape</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="k">const</span> <span class="kt">int</span> <span class="n">N</span> <span class="o">=</span> <span class="n">input1_shape</span><span class="p">.</span><span class="n">dim_size</span><span class="p">(</span><span class="mi">0</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">    <span class="k">const</span> <span class="kt">int</span> <span class="n">M</span> <span class="o">=</span> <span class="n">input2_shape</span><span class="p">.</span><span class="n">dim_size</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">    <span class="n">output_shape</span><span class="p">.</span><span class="n">AddDim</span><span class="p">(</span><span class="n">N</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">    <span class="n">output_shape</span><span class="p">.</span><span class="n">AddDim</span><span class="p">(</span><span class="n">M</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">    <span class="n">Tensor</span><span class="o">*</span> <span class="n">output_tensor</span> <span class="o">=</span> <span class="nb">NULL</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">    <span class="n">OP_REQUIRES_OK</span><span class="p">(</span><span class="n">context</span><span class="p">,</span> <span class="n">context</span><span class="o">-&gt;</span><span class="n">allocate_output</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">output_shape</span><span class="p">,</span> <span class="o">&amp;</span><span class="n">output_tensor</span><span class="p">));</span>
</span></span></code></pre></div><h4 id="实现矩阵乘法">实现矩阵乘法</h4>
<p>根据下面公式实现矩阵乘法</p>
<p>$$ { output_{ij} = input1_{ik} \times input2_{kj} }$$</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-cpp" data-lang="cpp"><span class="line"><span class="cl">    <span class="k">auto</span> <span class="n">input1</span> <span class="o">=</span> <span class="n">input_tensor1</span><span class="p">.</span><span class="n">matrix</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">();</span>
</span></span><span class="line"><span class="cl">    <span class="k">auto</span> <span class="n">input2</span> <span class="o">=</span> <span class="n">input_tensor2</span><span class="p">.</span><span class="n">matrix</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">();</span>
</span></span><span class="line"><span class="cl">    <span class="k">auto</span> <span class="n">output</span> <span class="o">=</span> <span class="n">output_tensor</span><span class="o">-&gt;</span><span class="k">template</span> <span class="n">matrix</span><span class="o">&lt;</span><span class="n">T</span><span class="o">&gt;</span><span class="p">();</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1">// matmul
</span></span></span><span class="line"><span class="cl"><span class="c1"></span>    <span class="k">for</span><span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o">&lt;</span> <span class="n">N</span><span class="p">;</span> <span class="n">i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">      <span class="k">for</span><span class="p">(</span><span class="kt">int</span> <span class="n">j</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">j</span> <span class="o">&lt;</span> <span class="n">M</span><span class="p">;</span> <span class="n">j</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="n">output</span><span class="p">(</span><span class="n">i</span><span class="p">,</span><span class="n">j</span><span class="p">)</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">        <span class="k">for</span><span class="p">(</span><span class="kt">int</span> <span class="n">k</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">k</span> <span class="o">&lt;</span> <span class="n">input1_shape</span><span class="p">.</span><span class="n">dim_size</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span> <span class="n">k</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">          <span class="n">output</span><span class="p">(</span><span class="n">i</span><span class="p">,</span><span class="n">j</span><span class="p">)</span> <span class="o">+=</span> <span class="n">input1</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">k</span><span class="p">)</span> <span class="o">*</span> <span class="n">input2</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">j</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">        <span class="p">}</span>
</span></span><span class="line"><span class="cl">      <span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span></code></pre></div><h4 id="添加约束条件">添加约束条件</h4>
<p>添加约束条件主要考虑到两点：</p>
<ol>
<li>自定义的op可能有多种实现，比如针对CPU和GPU有不同的实现</li>
<li>定义了多态，需要向TensorFlow系统指明本次注册的op实现是针对哪一种类型的</li>
</ol>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-cpp" data-lang="cpp"><span class="line"><span class="cl"><span class="cp">#define REGISTER_KERNEL(type)                                     \
</span></span></span><span class="line"><span class="cl"><span class="cp">  REGISTER_KERNEL_BUILDER(                                        \
</span></span></span><span class="line"><span class="cl"><span class="cp">    Name(&#34;Mymatmul&#34;).Device(DEVICE_CPU).TypeConstraint&lt;type&gt;(&#34;T&#34;),\
</span></span></span><span class="line"><span class="cl"><span class="cp">    MymatmulOp&lt;type&gt;)
</span></span></span><span class="line"><span class="cl"><span class="cp"></span>
</span></span><span class="line"><span class="cl">  <span class="n">REGISTER_KERNEL</span><span class="p">(</span><span class="n">int32</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="n">REGISTER_KERNEL</span><span class="p">(</span><span class="n">int64</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="n">REGISTER_KERNEL</span><span class="p">(</span><span class="kt">float</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="n">REGISTER_KERNEL</span><span class="p">(</span><span class="kt">double</span><span class="p">)</span>
</span></span></code></pre></div><p>这里定义了<code>REGISTER_KERNEL</code>宏，方便我们注册多种类型的op</p>
<h3 id="构建库文件">构建库文件</h3>
<p>本文提供了g++的构建方式，可使用下面的命令构建op的库文件</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="nv">TF_CFLAGS</span><span class="o">=(</span> <span class="k">$(</span>python -c <span class="s1">&#39;import tensorflow as tf; print(&#34; &#34;.join(tf.sysconfig.get_compile_flags()))&#39;</span><span class="k">)</span> <span class="o">)</span>
</span></span><span class="line"><span class="cl"><span class="nv">TF_LFLAGS</span><span class="o">=(</span> <span class="k">$(</span>python -c <span class="s1">&#39;import tensorflow as tf; print(&#34; &#34;.join(tf.sysconfig.get_link_flags()))&#39;</span><span class="k">)</span> <span class="o">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">g++ -std<span class="o">=</span>c++11 -shared op_mymatmul.cc -o op_mymatmul.so -fPIC <span class="si">${</span><span class="nv">TF_CFLAGS</span><span class="p">[@]</span><span class="si">}</span> <span class="si">${</span><span class="nv">TF_LFLAGS</span><span class="p">[@]</span><span class="si">}</span> -O2
</span></span></code></pre></div><p><code>TF_CFLAGS</code>和<code>TF_LFLAGS</code>分别为构建op所需要的头文件路径和库文件路径</p>
<h3 id="验证op可行性">验证op可行性</h3>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">tensorflow</span> <span class="k">as</span> <span class="nn">tf</span>
</span></span><span class="line"><span class="cl"><span class="n">m</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">load_op_library</span><span class="p">(</span><span class="s1">&#39;./op_mymatmul.so&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">a</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">constant</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="p">[[</span><span class="mf">1.</span><span class="p">,</span> <span class="mi">2</span><span class="p">],</span>
</span></span><span class="line"><span class="cl">        <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">],</span>
</span></span><span class="line"><span class="cl">        <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">]])</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">b</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">constant</span><span class="p">(</span>
</span></span><span class="line"><span class="cl">        <span class="p">[[</span><span class="mf">1.</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">1</span><span class="p">],</span>
</span></span><span class="line"><span class="cl">        <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">1</span><span class="p">]])</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="k">with</span> <span class="n">tf</span><span class="o">.</span><span class="n">Session</span><span class="p">(</span><span class="s1">&#39;&#39;</span><span class="p">)</span> <span class="k">as</span> <span class="n">s</span><span class="p">:</span>
</span></span><span class="line"><span class="cl">    <span class="nb">print</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">m</span><span class="o">.</span><span class="n">mymatmul</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">)))</span>
</span></span><span class="line"><span class="cl">    <span class="nb">print</span><span class="p">(</span><span class="n">s</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">tf</span><span class="o">.</span><span class="n">matmul</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">)))</span>
</span></span></code></pre></div><p><code>mymatmul</code>和TensorFlow自带的<code>matmul</code>输出结果一致。</p>
<h2 id="为op添加梯度计算">为op添加梯度计算</h2>
<p>如果将构建好的op应用到tensorflow搭建好的神经网络中，比如<code>mnist手写数字识别</code>，我们将得到梯度未定义的错误</p>
<blockquote>
<p>LookupError: No gradient define for operation &lsquo;Mymatmul&rsquo; (op type Mymatmul)</p>
</blockquote>
<p>因此我们还需要为op添加梯度计算</p>
<h3 id="注册op">注册op</h3>
<p>与前面流程类似，我们仍然需要先注册梯度op</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-cpp" data-lang="cpp"><span class="line"><span class="cl"><span class="n">REGISTER_OP</span><span class="p">(</span><span class="s">&#34;MymatmulGrad&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">.</span><span class="n">Attr</span><span class="p">(</span><span class="s">&#34;T: {float, int32, int64, double}&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">.</span><span class="n">Input</span><span class="p">(</span><span class="s">&#34;grad: T&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">.</span><span class="n">Input</span><span class="p">(</span><span class="s">&#34;input1: T&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">.</span><span class="n">Input</span><span class="p">(</span><span class="s">&#34;input2: T&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">.</span><span class="n">Output</span><span class="p">(</span><span class="s">&#34;grad_input1: T&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">  <span class="p">.</span><span class="n">Output</span><span class="p">(</span><span class="s">&#34;grad_input2: T&#34;</span><span class="p">);</span>
</span></span></code></pre></div><p>这里会接受三个输入，<code>grad</code>为矩阵乘法op输出的梯度，<code>input1</code>和<code>input2</code>为参与矩阵乘法的两个矩阵，输出为两个矩阵的梯度</p>
<h3 id="实现op-1">实现op</h3>
<p>已知输出梯度，求输入梯度，经典的反向传播求梯度</p>
<p>假设输出误差为$L$，输出为$y$， 两个输入矩阵分别$W$和$x$</p>
<p>$$  y = Wx    $$</p>
<p>$$  \dfrac{\partial L}{\partial x} = \dfrac{\partial y}{\partial x} \dfrac{\partial L}{\partial y}  = W\dfrac{\partial L}{\partial y} $$</p>
<p>根据公式编写代码如下</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-cpp" data-lang="cpp"><span class="line"><span class="cl"><span class="k">template</span><span class="o">&lt;</span><span class="k">typename</span> <span class="n">T</span><span class="o">&gt;</span>
</span></span><span class="line"><span class="cl"><span class="k">class</span> <span class="nc">MymatmulGradOp</span> <span class="o">:</span> <span class="k">public</span> <span class="n">OpKernel</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl"><span class="k">public</span><span class="o">:</span>
</span></span><span class="line"><span class="cl">  <span class="k">explicit</span> <span class="n">MymatmulGradOp</span><span class="p">(</span><span class="n">OpKernelConstruction</span><span class="o">*</span> <span class="n">context</span><span class="p">)</span> <span class="o">:</span> <span class="n">OpKernel</span><span class="p">(</span><span class="n">context</span><span class="p">)</span> <span class="p">{}</span>
</span></span><span class="line"><span class="cl">  <span class="kt">void</span> <span class="nf">Compute</span><span class="p">(</span><span class="n">OpKernelContext</span><span class="o">*</span> <span class="n">context</span><span class="p">)</span> <span class="k">override</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">    <span class="c1">// create input tensor ...
</span></span></span><span class="line"><span class="cl"><span class="c1"></span>
</span></span><span class="line"><span class="cl">    <span class="c1">// create output tensor ...
</span></span></span><span class="line"><span class="cl"><span class="c1"></span>
</span></span><span class="line"><span class="cl">    <span class="c1">// init
</span></span></span><span class="line"><span class="cl"><span class="c1"></span>    <span class="k">for</span><span class="p">(</span><span class="kt">int</span> <span class="n">j</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">j</span> <span class="o">&lt;</span> <span class="n">K</span><span class="p">;</span> <span class="n">j</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">      <span class="k">for</span><span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o">&lt;</span> <span class="n">N</span><span class="p">;</span> <span class="n">i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="n">grad_input1</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">j</span><span class="p">)</span> <span class="o">=</span> <span class="mf">0.0</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">      <span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="k">for</span><span class="p">(</span><span class="kt">int</span> <span class="n">j</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">j</span> <span class="o">&lt;</span> <span class="n">M</span><span class="p">;</span> <span class="n">j</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">      <span class="k">for</span><span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o">&lt;</span> <span class="n">K</span><span class="p">;</span> <span class="n">i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="n">grad_input2</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">j</span><span class="p">)</span> <span class="o">=</span> <span class="mf">0.0</span><span class="p">;</span>
</span></span><span class="line"><span class="cl">      <span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">    <span class="c1">// matmul
</span></span></span><span class="line"><span class="cl"><span class="c1"></span>    <span class="k">for</span><span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o">&lt;</span> <span class="n">N</span><span class="p">;</span> <span class="n">i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">      <span class="k">for</span><span class="p">(</span><span class="kt">int</span> <span class="n">j</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">j</span> <span class="o">&lt;</span> <span class="n">M</span><span class="p">;</span> <span class="n">j</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">        <span class="k">for</span><span class="p">(</span><span class="kt">int</span> <span class="n">k</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">k</span> <span class="o">&lt;</span> <span class="n">K</span><span class="p">;</span> <span class="n">k</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
</span></span><span class="line"><span class="cl">          <span class="n">grad_input1</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">k</span><span class="p">)</span> <span class="o">+=</span> <span class="n">input2</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">j</span><span class="p">)</span> <span class="o">*</span> <span class="n">grad</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">j</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">          <span class="n">grad_input2</span><span class="p">(</span><span class="n">k</span><span class="p">,</span> <span class="n">j</span><span class="p">)</span> <span class="o">+=</span> <span class="n">input1</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">k</span><span class="p">)</span> <span class="o">*</span> <span class="n">grad</span><span class="p">(</span><span class="n">i</span><span class="p">,</span> <span class="n">j</span><span class="p">);</span>
</span></span><span class="line"><span class="cl">        <span class="p">}</span>
</span></span><span class="line"><span class="cl">      <span class="p">}</span>
</span></span><span class="line"><span class="cl">    <span class="p">}</span>
</span></span><span class="line"><span class="cl">  <span class="p">}</span>
</span></span><span class="line"><span class="cl"><span class="p">};</span>
</span></span></code></pre></div><p>给op添加约束条件以及构建库文件同前面一样</p>
<h3 id="注册梯度计算">注册梯度计算</h3>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># FILE: op_mymatmul_grad.py</span>
</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">tensorflow</span> <span class="k">as</span> <span class="nn">tf</span>
</span></span><span class="line"><span class="cl"><span class="kn">from</span> <span class="nn">tensorflow.python.framework</span> <span class="kn">import</span> <span class="n">ops</span>
</span></span><span class="line"><span class="cl"><span class="n">m</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">load_op_library</span><span class="p">(</span><span class="s1">&#39;./op_mymatmul_grad.so&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="nd">@ops.RegisterGradient</span><span class="p">(</span><span class="s2">&#34;Mymatmul&#34;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="k">def</span> <span class="nf">mymatmul_grad_cc</span><span class="p">(</span><span class="n">op</span><span class="p">,</span> <span class="n">grad</span><span class="p">):</span>
</span></span><span class="line"><span class="cl">    <span class="k">return</span> <span class="n">m</span><span class="o">.</span><span class="n">mymatmul_grad</span><span class="p">(</span><span class="n">grad</span><span class="p">,</span> <span class="n">op</span><span class="o">.</span><span class="n">inputs</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">op</span><span class="o">.</span><span class="n">inputs</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
</span></span></code></pre></div><p>这里需要注意，<code>@ops.RegisterGradient(...)</code>里面传的是自定义op的名字，而不是梯度op的名字，因为我们要将梯度和自定义op绑定在一起</p>
<h3 id="验证可用性">验证可用性</h3>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-python" data-lang="python"><span class="line"><span class="cl"><span class="c1"># ...</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="kn">import</span> <span class="nn">op_mymatmul_grad</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># ...</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="c1"># In addition to replacing matrix multiplication with mymatmul, just write your neural network model normally</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">m</span> <span class="o">=</span> <span class="n">tf</span><span class="o">.</span><span class="n">load_op_library</span><span class="p">(</span><span class="s1">&#39;./op_mymatmul.so&#39;</span><span class="p">)</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl"><span class="n">m</span><span class="o">.</span><span class="n">mymatmul</span><span class="p">(</span><span class="o">...</span><span class="p">)</span>
</span></span></code></pre></div><p>最终结果正常。</p>
<h2 id="项目地址">项目地址</h2>
<p><a class="link" href="https://github.com/StubbornVegeta/tensorflow-custom-op"  target="_blank" rel="noopener"
    >StubbornVegeta/tensorflow-custom-op</a></p>

</section>


    <footer class="article-footer">
    
    <section class="article-tags">
        
            <a href="/tags/tensorflow1.x/">tensorflow1.x</a>
        
            <a href="/tags/op/">op</a>
        
    </section>


    
    <section class="article-copyright">
        <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-copyright" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
  <path stroke="none" d="M0 0h24v24H0z"/>
  <circle cx="12" cy="12" r="9" />
  <path d="M14.5 9a3.5 4 0 1 0 0 6" />
</svg>



        <span>Licensed under CC BY-NC-SA 4.0</span>
    </section>
    

</footer>


    
        <link 
                rel="stylesheet" 
                href="https://cdn.jsdelivr.net/npm/katex@0.15.6/dist/katex.min.css"integrity="sha256-J&#43;iAE0sgH8QSz9hpcDxXIftnj65JEZgNhGcgReTTK9s="crossorigin="anonymous"
            ><script 
                src="https://cdn.jsdelivr.net/npm/katex@0.15.6/dist/katex.min.js"integrity="sha256-InsNdER1b2xUewP&#43;pKCUJpkhiqwHgqiPXDlIk7GzBu4="crossorigin="anonymous"
                defer
                >
            </script><script 
                src="https://cdn.jsdelivr.net/npm/katex@0.15.6/dist/contrib/auto-render.min.js"integrity="sha256-y39Mpg7V3D4lhBX4x6O0bUqTV4pSrfgwEfGKfxkOdgI="crossorigin="anonymous"
                defer
                >
            </script><script>
    window.addEventListener("DOMContentLoaded", () => {
        renderMathInElement(document.querySelector(`.article-content`), {
            delimiters: [
                { left: "$$", right: "$$", display: true },
                { left: "$", right: "$", display: false },
                { left: "\\(", right: "\\)", display: false },
                { left: "\\[", right: "\\]", display: true }
            ]
        });})
</script>
    
</article>

    

    <aside class="related-contents--wrapper">
    
    
        <h2 class="section-title"></h2>
        <div class="related-contents">
            <div class="flex article-list--tile">
                
                    
<article class="has-image">
    <a href="/p/tensorflow%E7%B3%BB%E5%88%97/">
        
        
            <div class="article-image">
                <img src="/p/tensorflow%E7%B3%BB%E5%88%97/tensorflow.8b9924fea9fff47a19016cdabc52c06d_huc7c90fd3e3b57824859e512aa7252246_23873_250x150_fill_box_smart1_3.png" 
                        width="250" 
                        height="150" 
                        loading="lazy"
                        alt="Featured image of post TensorFlow系列"
                        
                        data-hash="md5-i5kk/qn/9HoZAWzavFLAbQ==">
                
            </div>
        

        <div class="article-details">
            <h2 class="article-title">TensorFlow系列</h2>
        </div>
    </a>
</article>
                
            </div>
        </div>
    
</aside>

     
    
        
    <script src='//unpkg.com/valine@latest/dist/Valine.min.js'></script>

<div id="vcomments"></div>
<style>
    #vcomments .vcards .vcard {
        background-color: var(--card-background);
	    padding: 25px 25px 5px 25px;
	    border-radius: 3px;
	    margin-bottom: 15px;
	    transition: all .3s
    }

    #vcomments .vcards .vh .vquote .vcard {
	    padding: 10px 25px 0px 0px;
    }

    #vcomments .vcards .vcard .vh .vcard {
        border: none;
        box-shadow: none;
    }

    #vcomments .vcards .vcard .vquote .vimg {
        width: 50px;
        height: 50px;
    }

    #vcomments .vcards .vcard .vimg {
        width: 60px;
        height: 60px;
        float: left;
        border-radius: 5px;
        margin-right: 18px;
        border: 0px;
        padding: 0px;
        box-shadow: 0px 10px 20px rgba(0, 0, 0, 0.04), 0px 2px 6px rgba(0, 0, 0, 0.04), 0px 0px 1px rgba(0, 0, 0, 0.04);
    }

    #vcomments .vcards .vcard .vcontent p{
        font-size: 1.6rem;
        color: var(--card-text-color-main);
    }

    #vcomments .vcards .vcard .vcontent{
        padding-top: 0px;
    }
    
    #vcomments .vcount {
        padding: 0px;
        margin-bottom: 5px;
        margin-top: 20px;
        font-weight: 700;
        font-size: 1.4rem;
        color: var(--body-text-color);
        text-transform: uppercase;
    }

    #vcomments .vwrap .cancel-reply-btn {
        position: relative;
        
    }

    #vcomments .vreply-wrapper .vwrap {
        box-shadow: var(--shadow-l2);
        border: 1px solid rgba(143, 143, 143, 0.3);
        width: 95%;
    }

    #vcomments .vquote .vreply-wrapper .vwrap {
        width: 93%;
    }

    #vcomments .vwrap {
        background-color: var(--card-background);
        border: 0px solid #f0f0f0;
        border-radius: 3px;
        margin-bottom: 10px;
        overflow: hidden;
        position: relative;
        padding: 0px 0px 0px 0px;
        font-family: inherit;
        display: flex;
        flex-direction: column;
    }

    #vcomments .vwrap .cancel-reply {
        position: absolute;
        z-index:999;
        float: right;
        top: -20px;
        right: 0px;
    }

    #vcomments .vwrap .vheader.item3{
        order: 0;
    }

    #vcomments .vwrap .edit{
        order: 1;
    }

    #vcomments .vwrap .vinput.vpreview{
        order: 2;
    }

    #vcomments .vwrap .vemojis{
        order: 3;
    }

    #vcomments .vwrap .vrow{
        order: 4;
    }

    #vcomments .vwrap .vmark{
        order: 5;
    }

    #vcomments .vpanel {
        order: 0;
    }

    #vcomments .vload-top.text-center {
        padding-top: 20px;
        order: 1;
    }

    #vcomments .vcount {
        order: 2;
    }

    #vcomments .vcards {
        order: 3;
    }

    #vcomments .vload-bottom.text-center {
        order: 4;
    }

    #vcomments .vempty {
        order: 5;
    }

    #vcomments .vpage.txt-center {
        order: 6;
    }

    #vcomments .vpower.txt-right {
        order: 7;
    }
    
    #vcomments {
        display: flex;
        flex-direction: column;
    }

    #vcomments .vcards .vcard .vhead .vnick {
        color: var(--accent-color);
        position: relative;
        font-size: 1.6rem;
        font-weight: 700;
        margin-right: .875em;
        cursor: pointer;
        text-decoration: none;
        display: inline-block;
    }
    #vcomments .vcards .vcard .vh .vtime {
        color: var(--card-text-color-secondary);
        font-weight: lighter;
        font-size: 1.3rem;
        margin-right: .875em;
    }

    #vcomments .veditor {
        font-size: 1.6rem;
        color: var(--card-text-color-main);
        padding: 20px;
        max-height: 350px;
        resize: none;
    }
    
    #vcomments .vedit {
        border-bottom: 1px solid rgba(143, 143, 143, 0.3);
        background-color: rgba(255, 255, 255, 0.08);
        padding-top: 0px;
    }    

    #vcomments .vrow {
        padding: 20px;
    }

    #vcomments .vedit .vrow {
        padding: 0px 20px 5px 20px;
    }

    #vcomments .vicon.vemoji-btn {
        fill: var(--card-text-color-tertiary);
    }

    #vcomments .vicon.vpreview-btn {
        display:none !important;
    }

    #vcomments .vwrap .vemojis {
        padding: 20px;
    }

    #vcomments .vquote .vmeta .vat {
        right: 30px;
    }

    #vcomments .vmeta .vat {
        background: 0 0;
        cursor: pointer;
        border: 0;
        border-radius: 5px;
        text-transform: uppercase;
        color: rgba(255, 255, 255, 0.5);
        transition: box-shadow .3s ease;
        height: 60px;
        width: 60px;
        display: inline-flex;
        justify-content: center;
        align-items: center;
        position: absolute;
        top: -20px;
        right: 10px;
    }

    #vcomments .vmeta .vat svg {
        stroke-width: 1.33;
    }
    
    #vcomments .vmeta .vat:hover {
        outline: 0;
        box-shadow: var(--shadow-l2);
    }

    #vcomments .vheader {
        display: flex;
    }

    #vcomments .vheader p {
        display: flex;
        flex-direction: column;
        align-items: flex-start;
        flex: 1;
        position: relative;
        margin: 0;
    }

    #vcomments .vheader label {
        position: absolute;
        top: 10px;
        left: 20px;
        color: var(--card-text-color-tertiary);
        margin-top: -4px;
        font-size: 14px;
    }

    #vcomments .vheader input {
         
        color: var(--card-text-color-main);
        border: 0;
        background: 0 0;
        padding: 35px 20px 15px;
        width: 100%;
        line-height: 1;
        transition: border .3s ease;
        font-size: 100%;
    }

    #vcomments .vheader p:not(:last-of-type) {
        border-right: 1px solid rgba(143, 143, 143, 0.3);
    }
    

    #vcomments .vh .vhead {
        padding-top: 5px;
    }

    #vcomments .vh .vquote .vhead {
        padding-top: 0px;
    }

    #vcomments .vbtn {
        color: var(--card-text-color-main);
        font-weight: 700;
        font-size: 1.0em;
        border-radius: 3px;
        border-color: 1px solid #666666;
    }

    #vcomments .vmore.vbtn {
        font-size: 13px;
    }

    #vcomments .vbtn:hover {
        color: #bebebe;
        border-color: #707070;
    }

    #vcomments .vbtn:active {
        color: #bebebe;
        border-color: #707070;
    }

    #vcomments .vcards .vcard .vcontent.expand {
        cursor: auto;
        max-height: none;
        overflow: visible;
    }

    #vcomments .vcards .vcard .vcontent.expand::after {
        display:none !important;
    }

    #vcomments .vcards .vcard .vcontent.expand::before {
        
        display:none !important;
    }

    #vcomments .vsys {
        display:none !important;
    }

    #vcomments .vcards .vcard .vh {
        padding-bottom: 0px;
        border-bottom: 0px dashed #f5f5f5;
    }

    #vcomments .vcards .vcard .vquote {
        padding-top: 5px;
        padding-left: 15px;
        border-left: 0px;
        px solid rgba(187, 187, 187, 0.5);
    }

    #vcomments .vicon.actived {
        fill: var(--card-text-color-main);
    }


</style>

<script>
    
    let placeholder = `发表你的评论\n`

    var valine = new Valine();

    valine.init({
        el: '#vcomments',
        appId: 'KHkYnFUrXOhcAuIDuVXPRFRy-gzGzoHsz',
        appKey: 'VR5IeFKU36DHOY3tzXQo0aBM',
        avatar: 'robohash',
        placeholder: placeholder,
        enableQQ: true,
        lang:'zh-CN',
        requiredFields: [],
        recordIP: false,
        visitor: true
    })

    html = `<p><label>昵称（必填）</label><input name='nick' placeholder='张三' class='vnick vinput' type='text'></p>
            <p><label>邮箱</label><input name='mail' placeholder='joe@example.com' class='vmail vinput' type='email'></p>
            <p><label>网址</label><input name='link' placeholder='https://' class='vlink vinput' type='text'></p>`
            
    document.querySelector(".vheader").innerHTML = html
    document.querySelector(".vpower.txt-right").innerHTML = ""

    if(window.location.hash){
        var checkExist = setInterval(function() {
           if (document.getElementById(window.location.hash.substring(1))) {
              document.getElementById(window.location.hash.substring(1)).scrollIntoView({behavior: "smooth", block: "center", inline: "nearest"})
              clearInterval(checkExist);
           }
        }, 100);
    }

</script>


    

    <footer class="site-footer">
    <section class="copyright">
        &copy; 
        
            2022 - 
        
        2024 倔强的贝吉塔
    </section>
    
    <section class="powerby">
        Built with <a href="https://gohugo.io/" target="_blank" rel="noopener">Hugo</a> <br />
        主题 <b><a href="https://github.com/CaiJimmy/hugo-theme-stack" target="_blank" rel="noopener" data-version="3.7.0">Stack</a></b> 由 <a href="https://jimmycai.com" target="_blank" rel="noopener">Jimmy</a> 设计
    </section>
</footer>


    
<div class="pswp" tabindex="-1" role="dialog" aria-hidden="true">

    
    <div class="pswp__bg"></div>

    
    <div class="pswp__scroll-wrap">

        
        <div class="pswp__container">
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
        </div>

        
        <div class="pswp__ui pswp__ui--hidden">

            <div class="pswp__top-bar">

                

                <div class="pswp__counter"></div>

                <button class="pswp__button pswp__button--close" title="Close (Esc)"></button>

                <button class="pswp__button pswp__button--share" title="Share"></button>

                <button class="pswp__button pswp__button--fs" title="Toggle fullscreen"></button>

                <button class="pswp__button pswp__button--zoom" title="Zoom in/out"></button>

                
                
                <div class="pswp__preloader">
                    <div class="pswp__preloader__icn">
                        <div class="pswp__preloader__cut">
                            <div class="pswp__preloader__donut"></div>
                        </div>
                    </div>
                </div>
            </div>

            <div class="pswp__share-modal pswp__share-modal--hidden pswp__single-tap">
                <div class="pswp__share-tooltip"></div>
            </div>

            <button class="pswp__button pswp__button--arrow--left" title="Previous (arrow left)">
            </button>

            <button class="pswp__button pswp__button--arrow--right" title="Next (arrow right)">
            </button>

            <div class="pswp__caption">
                <div class="pswp__caption__center"></div>
            </div>

        </div>

    </div>

</div><script 
                src="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe.min.js"integrity="sha256-ePwmChbbvXbsO02lbM3HoHbSHTHFAeChekF1xKJdleo="crossorigin="anonymous"
                defer
                >
            </script><script 
                src="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe-ui-default.min.js"integrity="sha256-UKkzOn/w1mBxRmLLGrSeyB4e1xbrp4xylgAWb3M42pU="crossorigin="anonymous"
                defer
                >
            </script><link 
                rel="stylesheet" 
                href="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/default-skin/default-skin.min.css"crossorigin="anonymous"
            ><link 
                rel="stylesheet" 
                href="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe.min.css"crossorigin="anonymous"
            >

            </main>
    
        <aside class="sidebar right-sidebar sticky">
            <section class="widget archives">
                <div class="widget-icon">
                    <svg xmlns="http://www.w3.org/2000/svg" class="icon icon-tabler icon-tabler-hash" width="24" height="24" viewBox="0 0 24 24" stroke-width="2" stroke="currentColor" fill="none" stroke-linecap="round" stroke-linejoin="round">
  <path stroke="none" d="M0 0h24v24H0z"/>
  <line x1="5" y1="9" x2="19" y2="9" />
  <line x1="5" y1="15" x2="19" y2="15" />
  <line x1="11" y1="4" x2="7" y2="20" />
  <line x1="17" y1="4" x2="13" y2="20" />
</svg>



                </div>
                <h2 class="widget-title section-title">目录</h2>
                
                <div class="widget--toc">
                    <nav id="TableOfContents">
  <ol>
    <li><a href="#为什么使用cpp新建op">为什么使用cpp新建op</a></li>
    <li><a href="#如何使用cpp新建op">如何使用cpp新建op</a></li>
    <li><a href="#新建矩阵乘法op">新建矩阵乘法op</a>
      <ol>
        <li><a href="#注册矩阵乘法op">注册矩阵乘法op</a></li>
        <li><a href="#实现op">实现op</a>
          <ol>
            <li><a href="#创建输入输出张量">创建输入输出张量</a></li>
            <li><a href="#实现矩阵乘法">实现矩阵乘法</a></li>
            <li><a href="#添加约束条件">添加约束条件</a></li>
          </ol>
        </li>
        <li><a href="#构建库文件">构建库文件</a></li>
        <li><a href="#验证op可行性">验证op可行性</a></li>
      </ol>
    </li>
    <li><a href="#为op添加梯度计算">为op添加梯度计算</a>
      <ol>
        <li><a href="#注册op">注册op</a></li>
        <li><a href="#实现op-1">实现op</a></li>
        <li><a href="#注册梯度计算">注册梯度计算</a></li>
        <li><a href="#验证可用性">验证可用性</a></li>
      </ol>
    </li>
    <li><a href="#项目地址">项目地址</a></li>
  </ol>
</nav>
                </div>
            </section>
        </aside>
    

        </div>
        <script 
                src="https://cdn.jsdelivr.net/npm/node-vibrant@3.1.6/dist/vibrant.min.js"integrity="sha256-awcR2jno4kI5X0zL8ex0vi2z&#43;KMkF24hUW8WePSA9HM="crossorigin="anonymous"
                
                >
            </script><script type="text/javascript" src="/ts/main.js" defer></script>
<script>
    (function () {
        const customFont = document.createElement('link');
        customFont.href = "https://fonts.googleapis.com/css2?family=Lato:wght@300;400;700&display=swap";

        customFont.type = "text/css";
        customFont.rel = "stylesheet";

        document.head.appendChild(customFont);
    }());
</script>

    </body>
</html>
